This invention relates to image processing methods for automatic alignment between image features and defined structures.
Many computer vision applications require the enhancement and detection of image features for objects of interest detection, measurement and/or classification. Application domain knowledge is available in most of the computer vision applications. The application domain knowledge can often be represented as structures of image features such as edges, lines and regions. The structure information can be well defined in industrial applications such as semiconductor manufacturing, electronic assembly or machine part inspections. In machine part inspections, most of the work-pieces have Computer Aided Design (CAD) data available that specifies its components as entities (LINE, POINT, 3DFACE, 3DPOLYLINE, 3DVERTEX, etc.) and blocks of entities. In biomedical or scientific applications, structure information can often be loosely defined. For example, a cell nucleus is round and different shapes differentiate different types of blood cells or chromosomes.
Application domain structure information is often encoded into parameters for image processing and measurements. Structure-guided methods are used to enhance and measure image features along the directions of the image structures of interest. These methods provide sub-pixel, high performance image feature extraction, enhancement and measurements as described in U.S. patent application Ser. No. 09/738,846 entitled, xe2x80x9cStructure-guided Image Processing and Image Feature Enhancementxe2x80x9d by Shih-Jong J. Lee, filed Dec. 15, 2000 and U.S. patent application Ser. No. 09/739,084 entitled, xe2x80x9cStructure Guided Image Measurement Methodxe2x80x9d, by Shih-Jong J. Lee et. al., filed Dec. 14, 2000 and U.S. Patent Application entitled, xe2x80x9cStructure-guided Automatic Learning for Image Feature Enhancementxe2x80x9d, by Shih-Jong J. Lee et. al., filed May 23, 2001. However, the processing and measurement results are dependent on the accuracy of the structure specifications. The results could be erroneous if the image features mismatch the defined structures.
A general-purpose computer vision system provides teaching functions that encode application domain structure information and processing algorithms into the system and application functions that process new images using the encoded structure information and processing algorithms. In the teaching phase, human error could cause mismatch of image features with defined structure. In the application phase, mismatch could occur due to imperfect repeatability of the stage or misplacement of the objects of interest. The mismatch could significantly degrade the effectiveness of a computer vision system.
Prior art relies on tight control of the alignment between structure specification and objects of interest. This approach is costly, is subject to error, and produces a non-robust result. This invention provides a method that automatically detects and compensates for misalignment between image features and defined structures.
It is an object of the invention to automatically align image features with defined structures. The method of this invention facilitates high quality, consistent and reliable image processing results.
Another objective of this invention is to allow a low skill operator to encode application domain structure into a structure-guided image processing system and to accept reasonable errors in accomplishing that task.
A further objective of this invention is to allow effective computer vision applications in a not well-controlled environment where accurate placement of objects of interest cannot be guaranteed.
Many computer vision applications require the enhancement and detection of image features for objects of interest detection, measurement and/or classification. Application domain knowledge is available in most of the computer vision applications. The application domain knowledge can often be represented as structures of image features such as edges, lines and regions. Structure-guided methods are used to enhance and measure image features of the image structures of interest. These methods provide sub-pixel, high performance image feature extraction, enhancement and measurement.
A general-purpose computer vision system provides teaching functions that encode application domain structure information and processing algorithms into the machine vision system and application functions that process new images using the encoded structure information and processing algorithms. In the teaching phase, input error could cause mismatch between image features and application domain structure information. In the application phase, mismatch could occur due to imperfect repeatability of the inspection stage or misplacement of the objects of interest. The invention method improves on the alignment of application domain structure information with image features, thereby enhancing accuracy, repeatability, and robustness of objects of interest detection, measurement and/or classification.
In a preferred embodiment of the invention, the image structure and measurement/detection targets are specified using a caliper approach. The method divides a defined structure region into mutually exclusive sub-regions. It performs robust structure-guided estimation within each sub-region and then performs a robust structure-guided estimation combining all sub-regions. The automatic alignment method of this invention includes a structure estimation step, an alignment decision step and a structure alignment step.